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CROSS REFERENCE TO RELATED APPLICATIONS 

This appUcation claims priority of U.S. Provisional Application No. 60/447,360 
(Attorney Docket No. KLA1P112P), filed on 12 February 2003, which application is 
herein incorporated by reference in its entirety for all purposes. 



BACKGROUND OF THE INVENTION 

The present invention relates generally to inspection of semiconductor devices, 
such as test structures and other types of semiconductor structures. More specifically, it 
relates to techniques for classifying defects found on integrated circuit devices. 

Semiconductor defects may include structural flaws, residual process material and 
other siurface contamination which occur during the production of semiconductor wafers. 
Defects are typically detected by a class of instruments called inspection tools. Such 
instruments automatically scan wafer surfaces and detect, and record the location of 
anomalies using a variety of techniques. This information, or "defect map," is stored in a 
computer file and sent to a defect review station. 
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Using the defect map to locate each defect, a human operator observes each defect 
under a microscope and classifies each defect according to class (e.g., particle, pit, 
scratch, or contaminant). Information gained from this process is used to correct the 
source of defects, and thereby improve the efficiency and yield of the semiconductor 
production process. Problems with this classification method include the technician's 
subjectivity in identifying the defect class, and the fatigue associated with the highly 
repetitive task of observing and classifying these defects. 

Methods of automatically classifying defects, collectively known as Automatic 
Defect Classification or "ADC," have been developed to overcome the disadvantages of 
manual defect classification. A conventional ADC system uses image processing 
techniques to first detect the defect and then to classify the defect according to the 
defect's physical characteristics and background geometry. Comparing these physical 
characteristics to the physical characteristics of pre-classified defects in a training set 
permits automated defect classification. 

While this system reduces technician fatigue and increases the number of defects 
that can be classified per unit time once the training set has been generated, such training 
set programs sometimes fail to provide an accurate classification for some defects. The 
setup of the training set typically is time consuming because it requires the manual 
classification of thousands of defects. During this manual classification process, the user 
is often presented with thousands of substantially similar defects which require individual 
manual classification. Needless to say, this process requires a significant amount of man- 
hours for a user to set up the training set. 
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Accordingly, there is a need for improved mechanisms for more efficiently setting 
up an automatic defect classification system. Additionally, there is a need for optimizing 
and efficiently maintaining an existing classification system. 
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SUMMARY OF THE INVENTION 

Accordingly, mechanisms are provided for efficiently setting up and maintaining 
a defect classification system. In general terms, the setup procedure optionally includes 
automatically grouping a set of provided defects (e.g., defect images) and presenting a 
5 representative set fi-om each defect group to the user for classification. Altematively, a 
representative set from the whole defect set may be presented to the user for classification 
without first grouping the defects into groups. The representative set does not include all 
of the defects and is selected to optimize manual classification efficiency. After the 
initial manual classification of the representative defects, the setup procedure includes an 

10 automatic procedure for classifying the non-reviewed or unclassified defects based on the 
manual class codes fi-om the user-reviewed defects. After the automatic classification 
operation, the user may also be presented with defects fi-om each class which may require 
re-classification. In particular embodiments, the user is iteratively presented with defects 
which have classifications that are suspect, which are near classification boundaries, or 

15 have classifications that have a low confidence level until each class is pure or contains a 
same type of defect classes as assigned by the user. 

In one embodiment, a method of setting up an automatic defect classifier system 
for classifying semiconductor defects. Defect image data is provided, e.g., firom an defect 
review system. The defect image data is then grouped (e.g., using a natural grouping 
20 procedure) into a plurality of groups of one or more defects. A representative set of 
defects fi^om each group is then determined so as to optimize manual classification. The 
representative set of defects firom each group and not the defects which are not part of the 
representative set fi-om each group are then presented to a user for manual classification. 
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The defects which are not part of the representative set from each group are defined as 
non-reviewed defects. 

In a further embodiment, the following method operations are perfomied: (a) 
determining a probable class for each non-reviewed defect based on the manual 
5 classifications by the user (e.g., using a nearest neighbor procedure); (b) determining a 
representative set of defects from each probable class which include defects having a 
probable class with a lowest confidence level within such probable class; (c) presenting 
the representative set of defects from each probably class to the user for possible re- 
classification; and (d) repeating steps (a) through (c) until all defects within the 
10 representative set of defects from each probable class have a same class, wherein the 
repetition of the determination of the probable class is based on the manual classifications 
and the re-classification by the user. 

In another aspect, it is determined whether manual classification is present already 
for some of the defects, and the defects are grouped and the representative sets are 

15 presented only when there is no manual classification present. In one aspect, each 
representative set of defects is a manageable subset of defects from group's total defects. 
In another aspect, detemiining the representative set of defects for each group includes 
selecting only a single defect from among defects which are substantially similar to be 
included within the representative set for such group. In another implementation, 

20 determining the representative set of defects for each group includes selecting defects 
which are uniformly distributed among the group's defects to be included within the 
representative set for such group. In yet another aspect, determining the representative 
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set of defects for each group includes selecting defects which are the most diverse from 
the group's defects to be included within the representative set for such group. 

In yet another implementation, determining the representative set of defects for 
each group includes (1) when the defects for the each group total less than three, selecting 
all of the defects from the each group to be included within the representative set for such 
group; and (2) when the defects for the each group are equal to three or more, selecting 
defects based on the max/min algorithm until Va, of the defects from the each group are 
selected. 

In a further aspect, determining the representative set of defects from each 
probable class includes (1) determining a confidence level for each non-reviewed defect 
in each probable class; and (2) selecting the non-reviewed defects from each probable 
class which have the lowest confidence level for inclusion in the representative set for 
such probable class. In a specific implementation, the confidence level for each non- 
reviewed defect within each probable class is equal to a minimxmi distance between the 
each non-reviewed defect and two of the nearest classified defects divided by a maximum 
distance between the each non-reviewed defect and two of the nearest classified defects. 

In a fiirther embodiment, when there is manual classification present the method 
includes the operations (1) when there is a training set present, using the training set as a 
reference to detect new defects in other unclassified defects; (2) when there is not a 
training set present, using the classified defects as reference defects to detect new defects 
from other unclassified defects; (3) when new defects are found, grouping the new 
defects and presenting to the user for classification; and (4) when new defects are found, 
repeating operations (a) through (c) after the new defects are classified. 
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In another embodiment, a classifier system for classifying xmclassified defects is 
created when the user selects an option for creating the classifier system. Creating the 
classifier system is accomplished by repeating the operations for grouping, presenting the 
representative set from each group; and operations (a) through (c). The classified defects 
are used as reference defects for classifying unclassified defects as part of the classifier 
system, when the user selects a supervisor mode. Creating the classifier further includes 
using at least a portion of the classified defects as reference defects for classifying 
unclassified defects as part of the classifier system when the user selects an existing 
manual classification mode and using a grouping procedure for classifying unclassified 
defects as part of the classifier system when the user selects an unsupervised mode. 

In a further aspect, when the existing manual classification mode is selected, 
purity and accuracy matrices for a classifier are presented based on (1) an existing 
training set, (2) a training set selected using the max/min algorithm (3) a training set 
combining the existing training set and a training set selected using the max/min 
algorithm. A classifier is then created based on the user's selected from among the three 
classifiers. 

In another aspect, the invention pertains to a computer system operable to set up 
an automatic defect classifier system for classifying semiconductor defects data. The 
computer system includes one or more processors and one or more memory. In yet 
another aspect, the invention pertains to a computer program product for setting up an 
automatic defect classifier system for classifying semiconductor defects. The computer 
program product includes at least one computer readable medium and computer program 
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instructions stored within the at least one computer readable product configured to 
perform one or more of the above described inventive procedures. 

These and other features and advantages of the present invention will be presented 
in more detail in the following specification of the invention and the accompanying 
figures which illustrate by way of example the principles of the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a flowchart illustrating a procedure for increasing the efficiency of 
manual classification in setting up a classification system in accordance with one 
embodiment of the present invention. 

5 Figure 2 is a flowchart illustrating the operation of Figure 1 for grouping defects 

and presenting representative defects firom each group to a user for manual cleissification 
in accordance with embodiment of the present invention. 

Figure 3A is a screen shot showing representative sets of defects firom a selected 
number of groups. 

10 Figure 3B illustrates the application of the max/min algorithm to a plurality of 

defects in a particular group in order to select a representative set of defects fi-om such 
group in accordance with one embodiment of the present invention. 

Figure 4 is flowchart illustrating the operation of Figure 1 for tuning the seed set 
of classified defects in accordance with one embodiment of the present invention. 

15 Figure 5 illustrates a screen shot showing an example set of probably classes (or 

"probable bins") each having a set of representative defects presented to the user for 
possible reclassification. 

Figure 6 is a flowchart illustrating a procedure for creating a classifier in 
accordance with one embodiment of the present invention. 



ICLAlPn2 



10 



Figure 7 illustrates accuracy and purity matrices for an example (1) classifier 
based on the existing training set, (2) a classifier based on a training set selected using the 
max/min algorithm (3) a classifier based on a training set combining the existing training 
set and a training set selected using the max/min algorithm. 

5 Figure 8 is a screen shot of an accuracy and purity matrix for a classifier created 

based on a training set selected using the max/min algorithm. 

Figure 9 is a screen shot of an accuracy and purity matrix for an example of a 
classifier created based on a training set selected using the max/min algorithm with the 
manual class codes assigned by natural grouping algorithm. 

10 Figure 10 is a diagrammatic representation of a defect image analysis system in 

accordance with one embodiment of the present invention. 
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DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 

Reference will now be made in detail to a specific embodiment of the invention. 
An example of this embodiment is illustrated in the accompanying drawings. While the 
invention will be described in conjxmction with this specific embodiment, it will be 
understood that it is not intended to limit the invention to one embodiment. On the 
contrary, it is intended to cover alternatives, modifications, and equivalents as may be 
included within the spirit and scope of the invention as defined by the appended claims. 
In the following description, numerous specific details are set forth in order to provide a 
thorough understanding of the present invention. The present invention may be practiced 
without some or all of these specific details. In other instances, well known process 
operations have not been described in detail in order not to imnecessarily obscure the 
present invention. 

Figure 1 is a flowchart illustrating a procedure 100 for increasing the efficiency of 
manual classification in setting up a classification system in accordance with one 
embodiment of the present invention. Initially, defect data is provided in operation 101. 
Defect data is typically provided in the form of images obtained fi-om a high resolution 
review tool, such as an electron microscope. One example of a review tool is the eV300 
E-beam Defect Review tool available fi-om KLA-Tencor of San Jose, California. 
Typically, a semiconductor product, such as a product wafer or test wafer (or a device), is 
inspected for defects and a defect map is provided. The defect map is then used to obtain 
a high resolution image of each located defect, e.g., using the review tool. 

After defect image data is provided, it may then be determined whether there are 
any manual class codes present in operation 102. In other words, it is determined 
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whether a user has already manually classified a set of defects. If there are no manual 
class codes, a manual classification procedure will then be initiated in combination with 
an automatic process to increase the efficiency of such manual classification procedure. 
In general terms, techniques are provided so as to help a user efficiently classify a 
5 manageable subset of defects. First, the defects may be automatically grouped and a 
representative set of defects fi-om each group is presented to a user for classification in 
operation 104. However, this grouping operation is optionally and may be skipped. 
Alternatively, a representative set fi-om the whole defect set may be presented to the user 
for classification without first grouping the defects into groups. 

10 Figure 2 is a flowchart illustrating the operation 104 of Figure 1 for grouping 

defects and presenting representative defects from each group to a user for manual 
classification in accordance with embodiment of the present invention. Initially, the 
defects are automatically grouped into a selected number of groups in operation 202. In 
the illustrated embodiment, the user may select the number of groups. Alternatively, the 

15 number of groups may be pre-defined. Representative or seed defects are then presented 
from each group to the user for manual classification in operation 204. 

Any suitable grouping technique may be implemented to group or cluster similar 
defects. Several grouping techniques are fiirther described in U.S. Patent No. 5,991,699 
by Ashok V. Kulkami et al. issued 23 November 1999 and Intemational Application No, 
20 PCT/USOO/32635 filed 29 November 2000 published 7 June 2001, which are both 
incorporate herein by reference in their entirety for all purposes. 

In one example implementation, a natural grouping procedure or algorithm may 
be implemented to group the defects into a selected or predefined number of groups. A 
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natural grouping algorithm typically includes first obtaining a feature vector for each 
defect image. The feature vector may include any suitable number and type of 
quantifiable image characteristics, such as defect shape, defect size, defect intensity, 
background intensity, etc. The feature vectors are then plotted in N-dimensional space 
(where N is the nvunber of vector parameters) and spatially clustered or grouped based on 
each vectors relative position in such space. 

Figure 3A is a screen shot showing a representative sets of defects 304 firom a 
selected number of groups (Groups 1~5 are shown). The user may select or adjust the 
total number of groups by any suitable mechanism such as the illustrated slide bar 302. 
Other common input device types include an entry box for entering a specific number, a 
dial or knob for selecting a particular mmiber, and a set of buttons for selecting fi-om a 
plurality of fixed numbers. In the illustrated embodiment, the total number of groups 302 
is set to five by slider 302. 

Representative defects fi-om each group are presented to the user. Viewing the 
representative defects, the user may continue to adjust the total group number until he/she 
is satisfied that the presented defects are divided among a correct number of groups. For 
example, when each group has similar types of defects, the user may determine that total 
group nimiber is set correctly. Otherwise, when one or more group contain significantly 
differing defect types, the user may adjust the total group number. In the illustrated 
example, group 1 304a has 10 representative defects having a similar appearance; group2 
304b has 9 representative defects having a similar appearance; group3 304c has 10 
representative defects having a similar appearance; and group4 304d has 8 representative 
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defects having a similar appearance (groupS is not visible and may be seen by scrolling 
down). 

The representative defects generally include a minimum set of defects that require 
user classification. For instance, when there are a plurality of nearly identical defects 
5 within a group, a single defect from this identical set is selected to be included in the 
representative set for such group. Additionally, boundary defects or the most different 
defects within a group are typically selected as part of the representative set. 

Any suitable algorithm may be used to determine the representative defects for 
each group presented to the user for classification. The algorithm is selected to present a 
10 non-redundant and diverse set of defects for each group that maximizes diversity (/.e., 
presents the most diverse set of defects for each group). The representative set may also 
be selected to include defects which are uniformly distributed among the group's defects. 
In a specific implementation, the following algorithm is used to select the representative 
set for each group: 

15 (i) If the defect number for the group is less than 3, select all defects from such 

group; 

(ii) the maximum number of defects to be selected must be less than 10 defects 
for the group; 

(iii) select defects based on the following formula if condition (i) has not been met 
20 and (until condition (ii) has been reached or until Va of the defects for such group has 

been selected: maK{min(d^^...d„)}for all defects n , where d is the distance between a 
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defect that has yet not been selected as a representative defect and a representative defect, 
(herein referred to as the "max/min algorithm"). 

Figure 3B illustrates the application of the max/min algorithm to a plurality of 
defects in a particular group 352 in order to select a representative set of defects from 
such group in accordance with one embodiment of the present invention. Initially, a 
random defect 354 is selected. Next, the defect that is located a farthest distance (in 
feature space) from the initial defect 354 is selected as a representative defect. As shown, 
defect 366 is the defect located the farthest from the initial defect 354. The initial defect 
354 is then thrown out and not used in the representative set (at least initially). A second 
defect that is located the farthest distance from the first representative defect 366 is then 
selected. In the illustrated example, defect 368 is the second representative defect. 

The next defect which has the maximum minimum of the two distances to the two 
representative defects is next selected as part of the representative set. As shown, defect 
360 is located a distance dl from representative defect 368 and a distance d2 from 
representative defect 366. Likewise, defect 356 is located a distance dl from 
representative defect 368 and a distance d2 from representative defect 366, Defect 364 is 
located a distance dl from representative defect 368 and a distance d2 from 
representative defect 366. For each potential representative defect, a minimum distance 
(dl or d2) is determined. The minimum distance for defect 360 is dl; the minimum 
distance for 356 is dl, and the minimum distance for defect 364 is dl in the illustrated 
example. Out of these three potential representative defects 360, 356, and 364, defect 
364 has the maximum minimum distance (dl). That is, the minimum distance dl for 
defect 364 (to representative defect 366) is greater than both the minimum distance dl for 
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defect 360 (to representative defect 368) and the minimum distance dl for defect 356 (to 
representative defect 368). Accordingly, defect 364 is chosen as the third representative 
defect. The defect which has the maximum minimum distance to these three 
representative defects is then selected as the next representative defect (e.g., defect 360), 
and this process is repeated imtil a Ya of the group defects are selected or the maximum of 
10 is reached. 

In another technique, one may select the representative defects for a relatively 
tightly clustered group to include a defect in the center of the group and defects 
distributed along the outside border of the group at regular intervals. In more loosely 
clustered groups, one may select defects through the interior space which are evenly 
distributed at regular intervals from the center to the outside boxmdary. The intervals 
may be determined through experimentation to determine which interval values lead to 
more accurate classification schemes. 

Referring back to Figure 2, after manual classification, the manually classified 
defects are then added into a seed set in operation 210. The seed set is later used to tune 
the classification scheme of the defects as described fiirther below. 

After the representative defects from each group are manually classified by the 
user, the classification is then tuned by iteratively presenting representative defects to the 
user for classification in operation 105 of Figure 1. The initial set of manually classified 
defects (e.g., obtained in operation 210 of Figure 2) is used as the initial seed set. This 
seed set of classified defects is generally adjusted until it can be used to automatically 
and accurately classify other defects which have not been reviewed and manually 
classified by the user. 
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Figure 4 is flowchart illustrating the operation 105 of Figure 1 for tuning the seed 
set of classified defects in accordance with one embodiment of the present invention. In 
operation 401, the seed set initially includes the manually classified defects. The non- 
reviewed defects which have not been manually classified are automatically divided into 
a plurality of probable classes based on the cleiss codes manually assigned to the seed set 
defects in operation 402. Any suitable procedure may be implemented to classify defects 
based on an initial classified seed set of defects. For example, a neural net procedure or 
algorithm may be implemented. 

By way of another example, a nearest neighbor procedure or algorithm may be 
utilized. In general terms, an imknown input image is assigned a classification code 
equal to a defect having the most similar feature vector within the feature space. Several 
nearest neighbor embodiments are further described in U.S. Patent No. 6,104,836 by Ke 
Han issued 15 August 2000, which patent is incorporated herein by reference in its 
entirety for all purposes. 

Representative defects having the lowest confidence levels are then presented 
from each probable class to the user for possible re-classification in operation 404. In 
general terms, the boundary defects from each probable class are presented for possible 
re-classification. In a specific implementation, the following algorithm is used to 
determine the confidence level of each classified defect and the five defects having the 
lowest confidence value are then selected: 

confidence value = 1 - — (herein referred to as the "confidence algorithm") 
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where di represents the distance between the unknown defect and its nearest seed 
defect Ds and d2 represents the distance between the unknown defect and its second 
nearest seed defect which has a different manual code than the seed defect Dg. 

Figure 5 illustrates a screen shot showing an example set of probably classes (or 
5 "probable bins") 502 each having a set of representative defects presented to the user for 
possible reclassification. Any suitable input mechanism may be utilized to allow the user 
to reclassify the representative set of defects fi-om each probable class. As shown in 
Figure 5, the user interface includes a set of class identifiers 504, along with images of 
the representative defects from each probable class 502. Alternatively, the user may be 

10 sequentially presented with each probable group's representative defects for possible re- 
classification. The user may reclassify any of the represented defects 502 by selecting a 
new class 504 for any of defects 502. In a specific implementation, the user may select 
all or a portion from each probably bin 502 for placement into a particular class by 
selecting, dragging, and dropping the defects into the appropriate class identifier 504. 

15 The user may also change the number of classes by setting up a new class identifier. 

The illustrated interface for re-classifying representative defects from probable 
classes is not meant to limit the scope of the invention. Other types of interfaces are also 
contemplated. In another example, the interface may simply include an entry box for 
each defect in which the user may change the determined probable class code. For 
20 instance, a defect may be proximate to an entry box having a probable class code equal to 
"1", and the user may change this class code to a class code equal to "2." 

Referring back to Figure 4, afl;er the user reclassifies as necessary, it is then 
deteraiined whether all probable classes are still pure in operation 408. If the probable 
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classes are not pure, the newly classified defects are then added into the seed set in 
operation 426. That is, when the classes contain all the same class of manually classified 
defects, the class is deemed to be pure. The class number may also be updated, e.g., if 
changed by the user in operation 424. The manual class code of the seed defects may 
5 also be modified, e.g., if changed by the user, in operation 422. Probable classes may 
then be automatically determined again for the new seed defects and a representative set 
fi-om each probably class is presented to the user for possible reclassification in 
operations 402 and 404, respectively. These two operations are repeated until all the 
probable classes are determined to be pure. 

10 When all probable classes are pure, the newly classified defects are then added to 

the seed set in operation 416. Suspected misclassified defects are then presented to the 
user for possible re-classification in added to the seed set in operation 418. In one 
implementation, defect classifications which have a confidence level below a 
predetermined value are defined as suspected misclassified defects. Class codes are then 

15 assigned to non-reviewed defects based on manually assigned class codes of each 
probable class in operation 420 and the tuning procedure ends. 

Referring back to Figure 1, after the setup of the classification system is tuned, it 
may then be determined whether the user wishes to create a classifier in operation 120. 
For example, the user may be presented with the option of creating a classifier (e.g., a 
20 classifier creation input button). If a classifier is not to be created, the procedure 100 
ends. If a classifier is to be created, a classifier is created in operation 122. This 
classifier creation operation 122 generally includes creating a classifier based on one or 
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more portions of the manually classified defects and maintaining such classifier, which is 
described further below. The procedure 100 then ends. 

Referring back to operation 102 of Figure 1, if there are manual class codes 
already present, the defects may then be reviewed in operation 106, instead of proceeding 
through a manual classification process (operations 104 and 105). The user may then 
determine whether to reset the manual class codes in operation 108. If the manual class 
codes are to be reset, the procedure 100 goes to operation 104, where the defects are 
grouped again for user classification. 

If the manual class codes are not to be reset, it is then determined whether a 
training set is present in operation 110. If a training set is already present, the training set 
may be used as a reference to detect new types of defects in other non-reference defects 
in operation 112. If a training set is not present, the classified defects may be used as a 
reference to detect new defects in other non-reference defects in operation 114. In either 
case, it may then be determined whether new defects have been foimd in operation 116. 
If new defects have not been found, the classification scheme may be further tuned in 
operation 105. If new defects are foxmd, the new defects may then be grouped and 
manually classified in operation 118, and these classifications are then tuned in operation 
105. 

This new defect detection procedure is useful for classifier maintenance. New 
defects may be found by any suitable procedure using the provided training set. One 
example procedure includes: 

(i) A training set T and a defect set V (maybe fi-om a new wafer) are provided; 
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(ii) For every defect x in set V, let dt indicate the distance from x to T (minimum 
distance to every defect in T) and dv indicate the distance to its nearest neighbor in V. 

(iii) Let r = dv/dt. 

(iv) If dt > tl and r < t2, where tl and t2 are predefined fixed thresholds, x is 
defined as a new defect. 

After manual classification is complete, classifier creation may be initiated by the 
user in several different modes, e.g,, using the manually classified codes or not. Li the 
following illustrated embodiment, the user may choose between several different 
classifier creation techniques. Alternatively, the classification creation technique may be 
fixed. Figure 6 is a flowchart illustrating a procediwe for creating a classifier system in 
accordance with one embodiment of the present invention. Initially, it is determined 
whether manual class codes exist in operation 601. If manual class codes exist, it is then 
determined whether the existing classification scheme is to be used in operation 604. For 
example, the user may choose to use the existing classified defects as the classifier. 

If the existing classification scheme is to be used, it is then determined whether 
more defects need to be classified in operation 610. If no other defects need to be 
classified, it is then determined whether a training set is present in operation 606. A 
training set may have been created using conventional methods or may have been created 
using the manual classification and training set formation techniques of the present 
invention. If a training set is present, purity and accuracy data is determined and 
presented for three different classifiers in operation 608. The purity and accuracy data is 
generally determined by comparing training set defects to manually classified defects. 
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As shown in Figure 7, an accuracy and purity matrix is shown for a classifier based on 
the existing training set (702), a classifier based on training set selected through max/min 
algorithm (706), and a classifier based on a training set combining the existing training 
set and training set selected fi"om max/min algorithm (704). The user may then select the 
particular classifier to be used in operation 609. 

However, if there are no manual codes present, it is then determined whether a 
supervised mode is selected (e.g., by the user) in operation 602. If a supervised mode is 
selected, the manual classification process (operation 102 of Figure 1) is executed to 
produce classified defects according to inventive techniques of the present invention. A 
training set based on the classified defects is then formed in operation 613, and the 
training set is preferable formed based on implementing the max/min procedure on the 
classified defects. The user is then presented with an accuracy and purity matrix for the 
existing classification (and training set). Figure 8 is a screen shot of an accuracy and 
purity matrix for a classifier created based on a training set selected using the max/min 
algorithm. 

If the supervised mode is not selected, then an unsupervised mode is selected by 
defauh and a warning may be issued that the existing class codes and treiining set is going 
to be lost in operation 612. The user may be presented with an opportunity to prevent 
such loss and select a different classifier creation mode. After the waming is issued (and 
optionally the user has chosen to proceed), the defects are then grouped into groups (e.g,, 
by natural grouping) and each group is assigned a unique class code in operation 616. A 
training set may then be formed based on the classified defects or by using the max/min 
procedure on the classified defects. The user may then be presented with an accuracy and 
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purity matrix for the unsupervised classifier. Figure 9 is a screen shot of an accuracy and 
purity matrix for an example of a classifier created based on a training set selected using 
the max/min algorithm with the manual class codes assigned by natural grouping 
algorithm. 

Referring back to operation 610, if new defects need to be classified, the manual 
classification process (operation of Figure 1) is executed and then operations 613 and 614 
of Figure 6 are executed. 

The present invention may be implemented in any suitable combination of 
hardware and software. Such a system would likely include one or more processors and 
memory configured to implement the techniques of the present invention. Figure 10 is a 
diagrammatic representation of an electron beam review system in which the techniques 
of the present invention may be implemented. The detail in Figure 10 is provided for 
illustrative purposes. One skilled in the art would understand that variations to the 
system shown in Figure 10 fall within the scope of the present invention. For example. 
Figure 10 shows the operation of a particle beam (e,g,, electron beam) with a 
continuously moving stage. However, many of the classification setup techniques 
described herein are also useful in the context of other testing devices, including particle 
beams operated in step and repeat mode. As an altemative to moving the stage with 
respect to the beam, the beam may be moved by deflecting the field of view with an 
electromagnetic lens. Alternatively, the beam column to be moved with respect to the 
stage. 

Sample 1057 can be secured automatically beneath a particle beam 1020. The 
particle beam 1020 can be a particle beam such as an electron beam. The sample handler 
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1034 can be configured to automatically orient the sample on stage 1024. The stage 
1024 can be configured to have six degrees of fi'eedom including movement and rotation 
along the x-axis, y-axis, and z-axis. In one embodiment, the stage 1024 is aligned 
relative to the particle beam 1020 so that the x-directional motion of the stage is 
corresponds to an axis that is perpendicular to a longitudinal axis of inspected conductive 
lines. Fine alignment of the sample can be achieved automatically or with the assistance 
of a system operator. The position and movement of stage 1024 during the analysis of 
sample 1057 can be controlled by stage servo 1026 and interferometers 1028. 

While the stage 1024 is moving in the x-direction, the inducer 1020 can be 
repeatedly deflected back and forth in the y direction. According to various 
embodiments, the inducer 1020 is moving back and forth at approximately 100 kHz. 
According to a preferred embodiment, the stage 1024 is groxmded to thereby ground the 
substrate and any structure tied to the substrate (e.g., source and drains) to allow voltage 
contrast between the floating and grounded structures as the result of scanning the 
targeted features. 

A detector 1032 can also be aligned alongside the particle beam 1020 to allow 
fiirther defect detection capabilities. The detector 1032 as well as other elements can be 
controlled using a controller 1050. Controller 1050 may include a variety of processors, 
storage elements, and input and output devices. The controller may be configured to 
implement the classification setup techniques of the present invention. In one 
embodiment, the controller is a computer system having a processor and one or more 
memory devices. 
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Regardless of the controller's configuration, it may employ one or more memories 
or memory modules configured to store data, program instructions for the general- 
purpose inspection operations and/or the inventive techniques described herein. The 
program instructions may control the operation of an operating system and/or one or 
more applications, for example. The memory or memories may also be configured to 
store images of scanned samples, reference images, defect classification and position 
data, as well as values for particular operating parameters of the inspection system. 

Because such information and program instructions may be employed to 
implement the systems/methods described herein, the present invention relates to 
machine readable media that include program instructions, state information, etc. for 
performing various operations described herein. Examples of machine-readable media 
include, but are not limited to, magnetic media such as hard disks, floppy disks, and 
magnetic tape; optical media such as CD-ROM disks; magneto-optical media such as 
floptical disks; and hardware devices that are specially configured to store and perform 
program instructions, such as read-only memory devices (ROM) and random access 
memory (RAM). The invention may also be embodied in a carrier wave traveling over 
an appropriate mediimi such as airwaves, optical lines, electric lines, etc. Examples of 
program instmctions include both machine code, such as produced by a compiler, and 
files containing higher level code that may be executed by the computer using an 
interpreter. 

Although the foregoing invention has been described in some detail for purposes 
of clarity of understanding, it will be apparent that certain changes and modifications may 
be practiced within the scope of the appended claims. Therefore, the described 
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embodiments should be taken as illustrative and not restrictive, and the invention should 
not be limited to the details given herein but should be defined by the following claims 
and their full scope of equivalents. 
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